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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re 

International Application of 
Weston, et al. 

International Application No. 
PCT/GB99/01574 

International Filing Date: 
17 May 1999 

VIDEO SIGNAL PROCESSING 

PRELIMINARY AMENDMENT 

BOX PCT 

Assistant Commissioner for Patents 
Washington, DC 20231 

Sir: 

Please amend the application as follows prior to calculation of the filing fees. 
IN THE CLAIMS 

In Claim 5, line 1, delete "any one of the preceding claims" and insert —Claim 1— . 
In Claim 6, line 1, delete "any one of Claims 1 to 4" and insert -Claim 1~. 
In Claim 7, line 1, delete "any one of the preceding claims" and insert —Claim 1— . 
In Claim 12, line 1, delete "any one of Claims 8 to 11" and insert -Claim 8-. 
In Claim 13, line 1, delete "any one of Claims 8 to 11" and insert —Claim 8~. 
In Claim 14, line 1, delete "any one of Claims 8 to 13" and insert -Claim 8-. 
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REMARKS 

The claims have been amended to remove multiple dependent claims and to 
conform to U.S. Patent Office practice. Please enter this amendment before calculating 
the filing fees. 



Respectfully submitted, 




Derek C. Stettner 
Reg. No. 37,945 



File No. 87805-9016 

Michael Best & Friedrich LLP 
100 East Wisconsin Avenue 
Milwaukee, WI 53202-4108 
(414) 271-6560 



-2- 



rci 14 NOV 2000 kt-gb**..™ 

Qft/?o o s % h 



This invention relates to video signal processing and is particularly 
concerned with non-linear filtering. 

5 It has been found that in a wide variety of video signal processes - 

including de-interlacing, decoding, enhancement, noise reduction, and 
standards conversion - considerable advantage can be secured by the use of 
complex non-linear filters. It has been found in particular that polynomial 
filters can be very useful. In many applications, quadratic behaviour in the 

10 filter is not sufficient and third or higher orders are typically necessary. Where 
real time operation is required, hardware implementations are usually 
essential and the hardware costs of such high order polynomial filters are 
substantial. 

It is an object of the present invention to provide improved methods 
15 and apparatus in video signal processing which offer third or higher order 
behaviour in a relatively simple filter architecture. 

Accordingly, the present invention consists, in one aspect, in a method 
of video signal processing, comprising the steps of conducting three linear 
filtering operations on an input video signal to produce three filtered signals, 
20 each linear filtering operation comprising the taking of a weighted sum of 

pixels; and multiplying together said three filtered signals to produce an output 
video signal. 

Suitably, the weighted sum is taken over pixels of the input video signal 
defined by a filter aperture and, preferably, all three linear filtering operations 
25 have the same fitter aperture. 

In one embodiment, for at least one linear filtering operation, the taking 
of a weighted sum of pixels includes the output pixel of the respective linear 
filtering operation. 

In another aspect, the present invention consists in apparatus for video 
30 signal processing comprising an input terminal for receiving an input video 
signal; first, second and third linear filters each connected with the input 
terminal and arranged to provide an output through taking a weighted sum of 
pixels; a first multiplier for multiplying together the respective outputs of the 
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first and second filters; and a second multiplier for multiplying together the 
' respective outputs of the first multiplier and the third filter to produce an output 
video signal. 

Advantageously, a filter is interposed between the output of the first 
5 multiplier and the second multiplier. 

Preferably, the apparatus further comprises a linear filter path 
connected with the input terminal, and a combiner for combining the outputs 
of the linear filter path with the output of said second multiplier. 

Suitably, a filter is interposed between the output of the second 
10 multiplier and said combiner. 

The invention will now be described by way of example with reference 
to the accompanying drawings in which: 

Figure 1 is a block diagram of video signal processing apparatus 
15 according to the invention, in the form of a vertical de-interlacing filter; 

and 

Figure 2 is a diagram similar to Figure 1, illustrating a modification. 

The example will be taken of a de-interiacer and, for reasons of clarity. 
20 a de-interlacer will be described that utilises only vertical information. It will be 
understood that horizontal and temporal information could be included in ways 



In Figure 1, the new architecture can be seen to consist of two signal 
paths. A linear signal path 10 contains a traditional, vertical, six tap, linear 
25 filter (h, ln ) which has a typical sin(x)/x structure. If this were to be used without 
the non-linear signal path it would produce reasonable pictures, but they 
would contain some artefacts due to the interpolation process, notably jagging 
on diagonal and curved edges. 

In the non-linear signal path 20, the output of two four point linear 
30 filters (hi and h 2 ) are multiplied together and passed through a two point linear 
filter (h 4 ). The output of this is then multiplied with the output of a five point 
linear filter (h 3 ). The resulting signal is filtered through another two point linear 
filter (h 5 ) before being added on to the linear path. Although in this case the 
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filter lengths are 4, 5 and 2, larger filters with more taps can be used to give 
better results. The lengths (or more generally, the sizes) of the filters need 
not be related and can be made larger or smaller to provide different trade- 
offs between quality and cost. 

5 It will be recognised that the arrangement of Figure 1 serves to 

generate the "missing" lines in a de-interlacing operation. These new lines 
will be combined in a multiplexer with the (suitably delayed) "original" lines. 

The filter coefficients can be selected by 'training 1 the filter on real 
pictures. In this example of de-interlacing, a still frame is taken and split into 

10 fields. A set of coefficients is used to estimate Field 2 from Field 1 and the 
mean squared error between the estimate of Field 2 and the original Field 2 is 
measured. A genetic algorithm can then be used to search the multi- 
dimensional filter space for the set of filter coefficients that gives the lowest 
mean squared error. 

15 if the described non-linear de-interiacer is tested on the EBU/SMPTE 

test picture "Girl with Toys", the non-linear path is found to reduce the average 
mean squared error by approximately 15% with respect to the linear filter. 
There is also a noticeable reduction in jagging. 

A polynomial filter with the same number of input pixel taps produces 

20 an almost equivalent reduction in error. However, a major advantage of this 
new architecture over the polynomial filter can be seen by considering the 
number of multiplications of pixels; multiplications of pixels by a constant; and 
additions, that each filter requires. These are shown in Table 1 . 





Polynomial 
Filter 


New 
Architecture 


Multiplication 


50 


2 


Multiplication by 
a constant 


34 


23 


Additions 


34 


24 



Table 1 : Comparison of complexity of filters 

25 

It can be seen that the largest reduction is in the multiplication of pixels. 
This is particularly significant as these are the most expensive to implement. 



In summary, the new architecture is able to reduce many of the 
artefacts associated with traditional linear interpolation whilst being relatively 
simple to implement. 

Figure 2 illustrates a modification in which the architecture is simplified 
through omission of the filters h 4 and h 5 . In other words, the direct product is 
formed of the outputs of filters hi, h 2 and h 3 without intervening filtering of the 
product of the outputs of filters hi and h 2 . This may under some 
circumstances produce less ideal filter behaviour, but the reduction in 
hardware complexity will often more than compensate. A particular 
advantage is that the three remaining filters can all make use of the same 
memory architecture. 

It should be understood that this invention has been described by way 
of example only and that a wide variety of modifications are possible without 
departing from the scope of the invention. Thus, whilst the separation into 
linear and non-linear paths offers important advantages, such as the option to 
preserve higher bit accuracy in the linear path, it will not always appropriate. 
Similarly, the described use of vertical filters is - as has been explained - 
merely an example. Horizontal, vertical and temporal filters can be employed 
and filters can have one, two or three of these dimensions. Whilst Finite 
Impulse Response (FIR) filters will be important, the invention also 
encompasses other forms of linear filter such as recursive filters which include 
the output Pixel in the weighted sum. The filters which are to be multiplied 
together need not be of the same category. However, providing three FIR or 
transversal filters with the same filter aperture ensures that in the 
multiplication of the three filtered signals, all possible cross products of input 
pixels are made available. 

It will be recognised that although de-interlacing has been chosen as 
an example, filters according to the present invention can be applied to other 
problems in video processing, including composite to component decoding, 
enhancement, noise reduction, up and down conversion and standards 
conversion -. 
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CLAIMS 

1 . A method of video signal processing, comprising the steps of conducting 
three linear filtering operations on an input video signal to produce three 
filtered signals, each linear filtering operation comprising the taking of a 
weighted sum of pixels; and multiplying together said three filtered signals 
to produce an output video signal. 

2. A method according to Claim 1 , wherein said weighted sum is taken over 
pixels of the input video signal defined by a filter aperture. 

3. A method according to Claim 2, wherein all three linear filtering operations 
have the same filter aperture. 

4. A method according to Claim 1 , wherein, for at least one linear filtering 
operation, the taking of a weighted sum of pixels includes the output pixel 
of the respective linear filtering operation. 

5. A method according to any one of the preceding claims, wherein the 
" product of two of said filtered signals is formed and a linear filtering 

operation conducted on that product, prior to multiplication of said product 
by the third filtered signal. 

6. A method according to any one of Claims 1 to 4, wherein said three filtered 
signals are multiplied together without intervening filtering of the three 
filtered signals. 

7. A method according to any one of the preceding claims, wherein a further 
linear filtering operation is conducted in parallel on the input video signal, 
with the result of said further linear filtering operation being combined with 
the multiplication product of said three filtered signals to produce an output 
video signal. 
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8. Apparatus for video signal processing comprising an input terminal for 
receiving an input video signal; first, second and third linear filters each 
connected with the input terminal and arranged to provide an output 
through taking a weighted sum of pixels; a first multiplier for multiplying 
together the respective outputs of the first and second filters; and a second 
multiplier for multiplying together the respective outputs of the first 
multiplier and the third filter to produce an output video signal. 

9. Apparatus according to Claim 9, wherein said weighted sum is taken over 
pixels of the input video signal defined by a filter aperture. 

10. Apparatus according to Claim 9, wherein said three linear filters have the 
same filter aperture. 

1 1 .Apparatus according to Claim 8, wherein at least one linear filter is 

arranged to take a weighted sum of pixels which includes the output pixel 
of the respective linear filter. 

12. Apparatus according to any one of Claims 8 to 1 1 , wherein there is further 
provided a linear filter connected between the output of said first multiplier 
and the input to said second multiplier. 

13. Apparatus according to any one of Claims 8 to 11 t wherein there is a direct 
connection between the output of said first multiplier and the input to said 
second multiplier. 

14. Apparatus according to any one of Claims 8 to 13, wherein the apparatus 
further comprises a linear filter path connected with the input terminal, and 
a combiner for combining the outputs of the linear filter path with the 
output of said second multiplier. 

15. Apparatus according to Claim 6, wherein a filter is interposed between the 
output of the second multiplier and said combiner. 
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fledaration apd Power of A ttorney For Patent Application 



As a below named inventor, I hereby declare that 

My residence, post office address and citizenship are as stated below next to 
my name, 

f believe I am an original, first and joint inventor of the subject matter which is 
claimed and for which a patent is sought on the invention entitled VIDEO SIGNAL 
PROCESSING (Attorney Docket No. 87805-9016), the specification of which was 
filed with my authority, on November 14, 2000 as Application Serial No. 09/700,321. 

I hereby state that I have reviewed and understand the contents of the above 
identified specification, including the claims referred to above, 

i acknowledge the duty to disclose to the Patent and Trademark Office all 
information known to me to be material to patentability as defined in TWe 37, Code of 
Federal Regulations, §1-56, 

As a named inventor, I hereby appoint the following registered practitioners to 
prosecute this application and transact all business in the Patent and Trademark 
Office connected therewith: 



Customer Number 



23409 



23409 



PATENT THADEHMtK OFFICE 



DIRECT ALL COMMUNICATIONS IN OR PERTAINING TO THIS 
APPLICATION TO: 



Customer Number 



23409 




PATENT TRMEHflRK OFFICE 



I hereby claim foreign priority benefits under Title 35, United States Code, 
§1 19 of the foreign application for patent or inventor's certificate listed below and 
have also identified below any foreign application for patent or inventor's certificate 
having 3 filing date before that of the application on which priority is claimed: 



The undersigned to this Declaration and Power of Attorney hereby authorize 
the U,S. attorneys named herein to accept and follow instructions from Mathys & 
Squire, 100 Gray's Inn Road, London WC1X SAL, England as to any actions to be 
taken in the U.S. Patent and Trademark Office regarding this application without 
direct communication between the U.S. attorneys and the undersigned, in the event 
of a change in the person(s) from whom instructions may be taken, the undersigned 
will so notify the U-S* attorneys. 

1 hereby dedare that all statements made herein of my own knowledge are 
true and that all statements made on information and belief are believed to be true; 
and further that these statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or Imprisonment, or both, 
under Section 1001 of Title 18 of the United States Code and that such willful fafse 
statements may jeopardize the validity of the application or any patent issued 
thereon, 



Full name of first joint inventor: Martin Weston 

Inventor's signature M H/^f ^x 

Date: t /x f <lqo % 



Prior Foreign Application 



(Number) 
PCT/GB99/01574 
9810555-4 



(Country) 
PCT 
United Kingdom 



(Day/Month/Year Filed) 
17 May 1999 
15 May 1998 



Citizenship: 

Post Office Address: 



Residence: 




7B Weston Road 

Petersfield, Hampshire GU31 4JF England 



Full name of second joint inventor William Beninfield Collis 



Inventor's signature ^ 
Date; 

Residence* 
Citizenship: 
Post Office Address: 



Southampton, Hampshire England 6>v^ 
Great Britain 

Lee's Cottage, 19 Jones Lane 

Hyvhe, Southampton, Hampshire S04S 6AW England 



J hereby claim foreign priority benefits under Title 35, United States Code, 
§1 19 of the foreign application for patent or inventor's certificate listed below and 
have also identified below any foreign application for patent or inventor's certificate 
having a filing date before that of the application on which priority is claimed: 

PriorForeign Application 

(Number) (Country) (Day/Month/Year Filed) 

PCT/GB99/01574 PCT 17 May 1999 

981 0555.4 United Kingdom 1 5 May 1 998 

The undersigned to this Declaration and Power of Attorney hereby authorize 
the U.S. attorneys named herein to accept and follow instructions from Mathys & 
Squire. 100 Gra/s fnn Road, London WC1X 8AL t England as to any actions to be 
taken in the U.S. Patent and Trademark Office regarding this application without 
direct communication between the US. attorneys and the undersigned. In the event 
of a change in the person(s) from whom instructions may be taken, the undersigned 
will so notify the U-S. attorneys. 

I hereby declare that ail statements made herein of my own knowledge are 
true and that all statements made on information and belief are believed to be true; 
and further that these statements were made with the knowledge that willful false 
statements and the Uke so made are punishable by fine or imprisonment, or both, 
under Section 1001 of Title 18 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or any patent issued 
thereon. 



Full name of first joint inventor: Martin Weston 

Inventor's signature 

Date: m 

Residence: Hampshire England 

Citizenship: Great Britain 

Post Office Address: 7B Weston Road 

Petersfield, Hampshire GU31 4JF England 



Full name of second joint inventor: ffiiliaw tes^gfeficid Coliis 

Inventor's signature /^J * 3 

Date: f£[tf*t 7 

Residence; SguttjahsfiSo^ HampshireEngland r/W*" 

Citizenship: Great Britain . 

Post Office Address; Lee*s Cottage. 19 Jones Lane 

Hythe, Southampton, Hampshire S045 SAW England 



